In search of on-line locality effects 1 Running head: IN SEARCH OF ON-LINE LOCALITY EFFECTS In Search of On-line Locality Effects in Sentence Comprehension

Many comprehension theories assert that increasing the distance between elements participating in a linguistic relation (e.g., a verb and an NP argument) increases the difficulty of establishing that relation during on-line comprehension. Such locality effects are expected to increase reading times, and are thought to reveal properties and limitations of the short-term memory system that supports comprehension. Despite their theoretical importance and putative ubiquity, however, evidence for on-line locality effects is quite narrow linguistically and methodologically: it is restricted almost exclusively to self-paced reading (SPR) of complex structures involving a particular class of syntactic relation. We present four experiments (two self-paced reading and two eyetracking experiments) that demonstrate locality effects in the course of establishing subject-verb dependencies; locality effects are seen even in materials that can be read quickly and easily. These locality effects are observable in the earliest possible eye-movement measures, and are of much shorter duration than previously reported effects. To account for the observed empirical patterns we outline a processing model of the adaptive control of button pressing and eye-movements. This model makes progress toward the goal of eliminating linking assumptions between memory constructs and empirical measures, in favor of explicit theories of the coordinated control of motor responses and parsing. In search of on-line locality effects 3 In Search of On-line Locality Effects in Sentence Comprehension One important goal of psycholinguistic research is to understand the memory processes that support the rapid comprehension of linguistic input, with its many temporally nonlocal relations. Both spoken and written comprehension require the comprehender to incrementally bring new input into contact with partial representations created on the basis of input that occurred earlier. This functional requirement for memory in the short term is easily seen in the nature of intrasentential linguistic relations such as those in (1) below, in which representations initially created upon reading or hearing the manager must be accessed at quit in order to establish the relationship between subject and verb: (1) a. The manager unexpectedly quit her job yesterday. b. The manager who the supervisor admired unexpectedly quit her job yesterday. The nature of the constraints and capacities of these memory processes has long been the topic of empirical and theoretical work in sentence processing, and current theories continue to advance a number of hypotheses about specific properties of this system, such as decay and similarity-based interference (Lewis & Vasishth, 2005; Gibson, 1998; Just & Carpenter, 1992; Gordon, Hendrick, Johnson, & Lee, 2006; Lewis, Vasishth, & Van Dyke, 2006). One of the most straightforward and theoretically influential empirical generalizations to emerge from this work is that the locality of linguistic relations, such as the subject-verb relation in (1) above, is a primary determinant of the speed and efficacy of the short-term memory processes in parsing (Chomsky, 1965; Just & Carpenter, 1992; Gibson, 1998). More specifically, increasing the distance over which these relations must In search of on-line locality effects 4 be computed degrades the underlying memory processes in some way. For example, the implication of this view for (1) is that the subject-verb relation in (1b) is more difficult to compute than the same relation in (1a). This theoretical view has been expressed most transparently in Dependency Locality Theory (DLT) (Gibson, 1998, 2000), which uses as a measure of locality the number of new linguistic referents interposed between a dependent and its head. DLT claims that the degree of locality should be reflected in a continuous and monotonic way in on-line reading time measures, thus yielding testable empirical predictions. We refer to this general class of effects on reading times as locality effects. While this paper presents evidence that locality effects are consistent with memory-based parsing theories, we call them locality effects without intending to associate them exclusively with the details of DLT or any other specific parsing model. Locality effects are important and relevant to a very broad range of extant memory and parsing theories (see Lewis et al. (2006) or Gibson (2000) for a summary)—even those which do not have mechanisms in place to directly produce them. Our aims for this paper are threefold. First, we briefly advance and defend the claim that current empirical evidence for on-line locality effects is narrow both linguistically and methodologically, and perhaps surprisingly difficult to find under the assumption that locality is a ubiquitous factor in sentence processing. More specifically, we raise the possibility that locality effects may be evident only in relatively complex structures whose difficulty may be traceable to independent factors. If this is the case, it has major implications for how these phenomena bear on theory development. Given the key role that locality effects play in shaping current parsing theory, we believe that it is important to significantly broaden its base of empirical support, and this relates to our second and third aims. Our second aim is to extend locality investigations to include eyetracking measures, which we will show has advantages over self-paced reading (SPR) for investigating locality effects. In search of on-line locality effects 5 Furthermore, we adopt an approach of running identical materials in both paradigms. This facilitates efforts to develop detailed theories of the link between the underlying short-term memory processes and the control of eye-movements and button-presses, and therefore the relationship between SPR and eyetracking as empirical measures. We sketch the beginnings of such a theory in the main discussion at the end of this paper. The model we propose awaits further empirical support, but it does capture key aspects of our results, and we believe holds promise of generalizing to other experiments. Our third aim is to demonstrate (possibly more subtle) locality effects using linguistic material that is, overall, significantly easier to process than materials that form the basis of existing locality demonstrations, thus providing stronger evidence for the claim that locality exerts pervasive and continuous effects on sentence processing. The remainder of this paper is structured as follows. We first provide our assessment of the current evidence for locality effects, and discuss its potential theoretical implications. We then describe the design and results from four new experiments, which consist of two pairs of SPR and eyetracking experiments. Finally, we discuss the theoretical and methodological implications of the results in the General Discussion, concluding with an outline of a theory of the adaptive control of eye-movements and button pressing that provides a framework for understanding the effects of underlying memory processes on the observable measures used in reading studies. Assessing current empirical evidence for locality effects The existing empirical evidence for locality effects is surprisingly mixed. Locality effects have been found in studies of English sentences (as we summarize below), but anti-locality effects—faster processing in longer-distance dependency integration—have been found in head-final languages including German, Hindi and Japanese (e.g., In search of on-line locality effects 6 Konieczny, 2000; Vasishth & Lewis, 2006; Vasishth, 2003; Nakatani & Gibson, 2008), as well as English (Levy, 2008, reporting an unpublished experiment by Jaeger and colleagues). Although anti-locality effects place important constraints on psycholinguistic parsing theory—and it is important to assess theories of locality effects in their context—it remains possible that independent factors give rise to both locality and anti-locality effects; they need not be mutually incompatible. Our concern in this paper is to develop a better understanding of the nature and extent of positive locality effects. In other work, we have outlined a theoretical model that provides an integrated explanation of both locality and anti-locality (Vasishth & Lewis, 2006; Lewis et al., 2006; Lewis & Vasishth, 2005). Locality effects have been observed in both ambiguous and relatively unambiguous structures. In ambiguous structures, locality plays a role in both resolving ambiguities (Kimball, 1973; Frazier & Fodor, 1978; Grodner, Gibson, & Tunstall, 2002; Gibson, Pearlmutter, Canseco-Gonzales, & Hickock, 1996; Pearlmutter & Gibson, 2001; Gibson, Pearlmutter, & Torrens, 1999; Altmann, Nice, Garnham, & Henstra, 1998) and in garden path reanalysis (garden paths involving longer ambiguous regions are typically more difficult to recover from; Pritchett, 1992; Gibson, 1991; Van Dyke & Lewis, 2003; Ferreira & Henderson, 1991). While these results have yielded useful constraints on parsing theory (Lewis & Vasishth, 2005), our present aim is to understand and find evidence for on-line locality effects in (putatively) globally unambiguous structures. (In the General Discussion we take up the issue of possible local ambiguity in our materials in some detail). Existing on-line locality effects are restricted to points of extraction Table 1 provides an overview of the existing experimental evidence for locality effects in relatively unambiguous structures. The evidence is restricted to English (a cross-linguistic gap that we do not fill in this paper), and to points of extraction—more In search of on-line locality effects 7 specifically, to relations conventionally analyzed as A-movement (of an argument) from its canonical position (Mahajan, 1990). In particular, the evidence generated so far involves relative clauses that contain so-called “filler-gap” dependencies (e.g., The man who the woman liked ), where the object has been displaced from its canonical position after the verb to the beginning of the sentence. It has been speculated in Grodner and Gibson (2005, p. 284) and elsewhere (Gibson, 2007) that A-movement may be an important condition for the occurrence of locality effects. Given this restricted evidential base, there are two plausible accounts for the locality effects that have been obtained experimentally. Locality effects may be a direct result of the degradation of memory representations between initial activation and subsequent retrieval for integration into a dependency, which would imply ubiquity of the effects. Alternatively, locality effects could reflect a source of difficulty unique to structures that require A-movement, such as object-extracted relative clauses. Although most theories of working memory in sentence processing do not distinguish the computational demands of movement and non-movement relations, there is a line of work that does make such a distinction, starting with the Hold Hypothesis in the augmented transition network (ATN) model of Wanner and Maratsos (1978), and continuing with the Grodzinsky (2000) theory of neural processes associated with syntactic movement operations. Prior experiments that could have determined if locality effects generalize beyond object relatives, and beyond movement, have yielded ambiguous results. The nature of the existing evidence can be understood by considering three of the experimental conditions in Grodner and Gibson (2005) Experiment 2 (underlining is used here to indicate the word at which the locality effects are predicted to be observed). Note that, in these sentences, A movement occurs when the object is moved from its base position (adjacent to the embedded verb) to the beginning of the sentence. In search of on-line locality effects 8 (2) Embedded verb conditions from Grodner and Gibson (2005) Experiment 2 a. The administrator who the nurse supervised scolded the medic while . . . b. The administrator who the nurse from the clinic supervised scolded the medic while. . . c. The administrator who the nurse who was from the clinic supervised scolded the medic while. . . In all three structures in (2a), the region of interest is the embedded verb supervised, and the locality manipulation involves increasing the distance from the embedded verb to its subject (the nurse) and its extracted object (the administrator). In (2a), no material intervenes between the embedded verb and the subject; in (2b), a three word prepositional phrase (PP) intervenes; and in (2c), a five word relative clause (RC) intervenes. The structure of this design is shown schematically in (3). The top arrow denotes the relation between the verb and the relative pronoun who that mediates the object extraction, and the bottom arrow denotes the subject relation. The ∅ symbol denotes the null string (nothing interposed). (3) Structure of the embedded verb conditions from Grodner and Gibson (2005) y The administrator who the nurse  ∅ from the clinic who was from the clinic  supervised. . . x The assumption (as expressed, e.g., in DLT) is that the computation of these dependency relations happens immediately at supervised by accessing short-term memory representations associated with the relativizing pronoun and the subject,1 and that this computation takes longer as the input items that triggered the target representations become more distant. Thus, the straightforward prediction is that reading times at In search of on-line locality effects 9 supervised should increase monotonically in the three conditions (nothing interposed, PP interposed, and RC interposed). This prediction is consistent with what Grodner and Gibson (2005) found in their Experiment 2 using self-paced reading, with the sharpest increase in reading times observed for the RC condition (we discuss the empirical results in more detail below). This manipulation has the attractive property that the specific verbs in the critical region and the head nouns of the target subject and object noun phrases are kept constant while changing the locality of the relations. But the reliable locality effect observed in (2) may have been driven entirely by the sharp increase in reading times for condition (2c): a case of double center-embedding of relative clauses, an effect that can be explained in ways that have nothing to do with locality (e.g., similarity-based interference, Lewis & Vasishth, 2005). How can we be sure that the observed effects in (3) generalize beyond object extractions over embedded relative clauses? We can compare the effects in (3) above to three other conditions in Grodner and Gibson (2005): (4) Matrix verb conditions from Grodner and Gibson (2005) Experiment 2 a. The nurse supervised the administrator while . . . b. The nurse from the clinic supervised the administrator while . . . c. The nurse who was from the clinic supervised the administrator while . . . These three conditions test for locality effects at a matrix verb from which no arguments have been extracted; the only linguistic relation affected by locality is the subject relation. The structure of the main verb conditions is shown schematically in (5): (5) Structure of the matrix verb conditions from Grodner and Gibson (2005) Experiment 2 In search of on-line locality effects 10 y The nurse  ∅ from the clinic who was from the clinic  supervised the administrator. . . If a locality effect is observed at supervised in (5), this would provide evidence that dependencies that are not the result of A-movement relations are also subject to locality effects.2 In other words, the presence of such effects in both kinds of structures would mean that increasing locality increases the processing cost of resolving simple subject-verb dependencies as well as object extractions. Figure 1 (upper left) shows the readings times observed by Grodner and Gibson (2005) at the critical verb. (This figure also contains the reading times for the four experiments in this paper, but the reader should focus for now on the upper-left graph). We can now ask whether these extant results help to extend the empirical base of locality effects beyond relative clauses. Unfortunately, they do not. Separate locality contrasts within the matrix verb condition were not reported in Grodner and Gibson (2005), but do not appear to be reliable. The contrast between the PP and no-interposition conditions in the embedded structures also was not reported, and also appears not to be reliable.3 In short, it is quite possible that the locality effects are driven by independent sources of difficulty resulting from embedding the verb and from center-embedding the relative clauses.4 Despite the ambiguity attending the Grodner and Gibson (2005) results, we believe that the structure of their Experiment 2 is still a promising way, in principle, to explore locality effects, and we adopt its structure for the four experiments presented here. But before moving on to the new experiments, we consider briefly the implications of the narrow methodological base for investigating locality effects. In search of on-line locality effects 11 A concern about the existing self-paced reading evidence for locality Self-paced reading has the virtue of yielding a simple measure that is often sensitive to the fluctuating processing demands of incremental comprehension. But because each word (or phrase) disappears as soon as the reader presses a button, the stakes of each button press are high relative to moving the eyes forward in reading. If the reader encounters difficulty that would best be resolved by regressing to an earlier part of the sentence, for instance to find a particular argument, he or she has no recourse in self-paced reading but to try to remember or mentally rehearse what came before. Eye-movements could potentially leave an interpretable record of such recovery processes, but SPR cannot—except perhaps in significantly increased reading times. This difference between SPR and eyetracking turns out to be crucial for interpreting SPR reading time data such as that in Grodner and Gibson (2005). The locality results observed by Grodner and Gibson (2005) are marked by an increase in reading times for the most difficult condition (the doubly embedded relative clause, (2c)). It is therefore possible that these effects reflect recovery from failed argument-verb integration caused by the center-embedding. More specifically, the observed 125–150ms increase in reading time may not be due to longer integration or memory processes affected by locality, but primarily recovery processes—perhaps covert rehearsal—triggered by retrieval failures5.To anticipate one of the findings reported in this paper: the combined results of our experiments provide support for this interpretation of existing SPR locality effects. Why does it matter whether observed effects are associated with recovery or initial retrieval or integration? It matters for the purpose of building a cumulative quantitative base of results on which to build computational theories of the underlying memory processes. We should, in principle, be able to use the empirical results from reading studies along with our developing models of memory in parsing to converge on stable estimates of memory retrieval processing rates that may be meaningfully compared (and In search of on-line locality effects 12 combined with) processing rate estimates obtained through other methodologies, such as speed-accuracy-tradeoff paradigms (McElree, Foraker, & Dyer, 2003). Such quantitative integration is important not simply because we desire quantitative predictions but because it facilitates theoretical integration. Overview of the empirical strategy and four experiments We now provide a brief overview of our empirical strategy and describe how it is realized in the four new experiments that follow. The overall goal is to determine if it is possible to observe locality effects that are not subject to the critiques above. Ideally, this means observing locality effects at points of computing relations that do not involve A movement or interference between multiple arguments, and observing locality effects under conditions of relatively easy processing. We employ four empirical devices to achieve these goals: 1. We adopt the six-condition structure of Grodner and Gibson (2005), outlined above in (2) and (4), which in principal has the potential to reveal locality effects in the main clause conditions at points that do not involve extraction. 2. We run eyetracking as well as SPR versions of each experiment. The specific aims are to (a) provide potentially more sensitive measures of locality effects in easy, non-extraction structures; (b) distinguish between locality effects on early measures (if they exist) vs. late measures in the eye-movement record; and (c) provide a better understanding of the nature of locality effects observed in SPR by providing evidence bearing on the specific hypothesis above concerning the role of parsing failure and recovery in SPR. 3. We adopt a new set of stimuli based on these structures but with content words drawn from a list of relatively short (three to six letter), high frequency words. The specific aims are to (a) increase the overall ease of processing and therefore provide an In search of on-line locality effects 13 additional test of the hypothesis that locality effects might only be evident in the presence of other sources of processing difficulty; (b) decrease item-dependent variance related to the length and frequency of content words; and (c) increase the proportion of single fixations in the eye-movement record which might provide the best opportunity to observe the early manifestations of locality. 4. In the new set of stimuli, we use only inanimate nouns in the extracted object position. As described above, both the subject and extracted object in the original Grodner and Gibson (2005) materials were noun phrases referring to humans. Thus in addition to increasing locality, the embedding manipulation also potentially increased similarity-based interference. The four experiments thus cross materials (original Grodner & Gibson stimuli and new stimuli) with method (SPR and eyetracking). Experiment 1 is SPR with the original Grodner & Gibson materials (a replication of their Experiment 2), Experiment 2 is eyetracking with the original materials, Experiment 3 is SPR with the new materials, and Experiment 4 is eyetracking with the new materials. For simplicity of presentation and analysis, we perform complete analyses on each experiment separately, but report a small number of key comparisons that test materials effects directly between Experiments 1 and 2, and 3 and 4. Experiment 1: Replication of Grodner & Gibson (2005) Exp. 2